The recognition of bending finger points based on video is difficult and recognition rate is not high. Focusing on the issue, a hand gesture recognition method based on depth image, skeleton image and color image information was proposed. Firstly, the gesture in color image area was initially determined rapidly by using depth information and skeleton information of Kinect, and the hand posture region was extracted from the gesture area by using the YCrCb color model. Then the distances between the gesture contour points and the palm point were calculated to generate the distance curve, and the ratio of curve peaks to troughs was set up to obtain finger point. Finally, commonly used 12 gestures were identified by combining bending finger point features and the maximum amount of contour area. Six experimenters were invited to validate the proposed method in experimental results verification phase. Every gesture was experimented 120 times under the condition of relatively stable light environment and the recognition rate of 12 gestures was 97.92% on average. The experimental results show that the proposed method can quickly location gestures and accurately recognize the commonly used 12 kinds of hand gestures, and the recognition rate is high.